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Abstract 

The unabashedly messy aspects of the research process are often hidden from 
published view, and are therefore not available to encourage and instruct. The 
authors tell specific stories about “messy” research, arranged around: 

• evolving research questions, 

• methodology or methods surprises, 

• problematic answers, and 

• publication dilemmas. 

These stories are from work amongst diverse categories and frames of 
inquiry, and are meant to encourage fellow students to persist and problem- 
solve mid-process. We hope that having real researchers tell their own sto- 
ries of mucking around the methodologies and ethics of inquiry can encour- 
age others to engage and persist, and can be the beginning of a more 
process-oriented and perhaps more nuanced conversation that opens active 
discussion and scrutiny amongst critical friends. 

Resume 

Tous les moyens sont bons pour que les aspects compliques du processus de 
recherche soient habituellement masques de la version publiee et ils nepeu- 
vent done pas servir a stimuler ou a former. Les auteurs racontent des his- 
toires particulieres au sujet de recherches compliquees qu’on adapte : 

• au developpement de questions de recherche 

• aux surprises rencontrees en methodologie ou dans la methode 

• aux reponses problematiques 

• aux dilemmes de publication 

Ces histoires provierment de travaux de categories et de cadres de recherche 
divers et sont sensees encourager des confreres a perseverer et a resoudre des 
problemes pendant la recherche. Nous esperons, qu’en ayantde vrais 
chercheurs qui racontent leurspropres histoires de decantation afin de reridre 
correctes les methodologies et Vethique de recherche, que cela en encouragera 
d’autres a s’engager et a persister etpeut etre initiera un processus plus 
methodique et possiblement un dialogue mieux nuance quipermettra une dis- 
cussion active et une recherche minutieuse avec des collegues critiques. 
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Introduction 

The error spectrum in publishable research, ranging from slight “unde- 
tectable” flaw to insurmountable foible, is known to exist, but is seldom 
acknowledged in print unless such an admission is followed by the victorious 
tale of how the blemish was erased, overcome, or adeptly sidestepped on the 
way to a research publication. This is not a new tradition; Figure 1 contains 
a 50-year-old tongue-in-cheek research glossary (from metallurgical research) 
of such sidestepping prose. 


Sidestepping Research Prose 

Common words and phrases 

Their real meanings 

It has long been known that. . . 

I haven 't bothered to look up the original 
reference. 

. . .of great theoretical and practical importance 

. . . interesting tome 

While it has not been possible to provide 
definite answers to these questions... 

The experiments didn 't work out. but I 
figured I could at least get a publication 
out of it. 

Three of the samples were chosen for detailed 
study... 

The results on the others didn 't make sense and 
were ignored. 

Typical results are shown. . . 

The best results are shown. 

Presumably at longer times. . . 

I didn't take time to find out. 

These results will be reported at a later date. . . 

I might possibly get around to this sometime. 

It is suggested that. . . It is believed that. . . 
It may be that. . . 

I think. . . 

It is generally believed that. . . 

A couple of other guys think so, too. 

It might be argued that. . . 

I have such a good answer to this objection that 
I shall now raise it. 

It is clear that much additional work will be 
required before a complete understanding. . . 

I don 't understand it... 

It is to be hoped that this work will stimulate 
further work in the field... 

This paper isn't very good, but neither are any 
of the others in this miserable subject. 


Figure 1. Sidestepping research prose (Graham, 1957). 

We believe that too little attention is given to acknowledging and (especially) 
disseminating the “stubbed toes” that happen on the research path. In this arti- 
cle, we seek not to sidestep but rather to highlight, and perhaps even celebrate, 
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the unabashedly messy business that is quality research. To do so, we tell sto- 
ries from our own experiences that illuminate our belief that research happens 
not at a failure/success binary/dichotomy, but rather along a path where many 
points can’t (and shouldn’t) be described as failure or success. 

The variety of experience level among authors, from highly published 
tenured professor to unpublished graduate student, is meant to show that this 
is true for researchers at every experience level, and that a key is to engage 
and persist. Writing often tidies up the loose and ugly ends of research, and 
yet the process of dealing with those parts is often one of the most produc- 
tive for a project. Our hope is that stories of our flaws and foibles will generate 
thought and provoke discussion. We wish for others to be heartened, and to 
see their own difficulties as both more useful to themselves, and perhaps 
instructive to be shared with others. 

The impetus for this paper was provided by a session held at the Third 
Annual Research Symposium in October 2006, prior to the annual conference 
of the North American Association for Environmental Education in St. Paul, 
Minnesota. The session, entitled “What Happens When Research Goes 
Bad?” featured a panel composed of Justin Dillon, Paul Hart, Joe Heimlich, and 
Michael Brody, and was moderated by Charlotte Clark. 

As we did at that conference session, we will parse our stories into 
four themes: 

• evolving research questions, 

• methodology or methods surprises, 

• problematic answers, and 

• publication dilemmas. 


Evolving Research Questions 

In qualitative work, we expect our research questions to evolve as we work 
and to be preconceived as emergent, rather than preordinate. However, the 
back story of that evolution in research work is seldom articulated, at least 
in print (however, see Lather & Smithies, 1997; Russell, 2003). We provide 
two such back stories here. In the first, the question evolved through exter- 
nal influence through a journal review process; in the second, the question 
evolved through internal inquiry. 

Welcoming External Influence on Research Questions [Michael and Justin] 

Michael: This tale emerged from a dynamic editorial process resulting in a 
career-defining article that led to further development of learning theory relat- 
ed to informal settings, and challenged me to grow professionally by leading 
me to greater insight and understanding that I otherwise would have over- 
looked (Brody, Tomkiewicz, & Graves, 2002). The text is presented as a 
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reflective dialogue on the editorial process and subsequent evolving world- 
view, research questions, research concept, and published paper. 

The research contract asked the authors to ascertain if a new Midway 
Geyser Basin Visitor Guide would have any effect on visitor outcomes; that 
is, would casual visitors learn anything as a result of using the brochure at the 
site. I went into this research project with an existing understanding of 
learning derived particularly from my previous work on misconceptions 
about the environment, and influenced primarily by Ausubel and Novak 
(Ausubel, Novak, & Hanesian, 1978). This led me to ask what people already 
know about Midway Geyser Basin and the associated life forms. 1 asked what 
science content visitors brought to the event, how that content might change, 
and what views and science misconceptions people might have about 

• the National Park Service, 

• geothermal features at the park, and 

• associated microorganisms (especially biopiracy of microorganisms for the 
biomedical industry (Brody et al., 2002)). 

Justin: Michael’s first draft was sent to two reviewers of the International 
Journal of Science Education that Bill Scott (co-editor) and I had asked to help 
with the special edition. The first review, undertaken by a leading researcher 
in science learning in informal contexts and completed in September 2000, 
concluded by stating that the paper: 

raises interesting questions, but needs to be more firmly grounded in relevant non- 
school based research and recent research on alternative conceptions. It also 
needs a more thorough defense of the research design and an introduction that 
sufficiently supports the study conducted. 

The second review, undertaken by a leading science education researcher, 
noted that the literature was somewhat out-of-date and suggested other 
studies should be used: 

Specifically, there are now strongly recognized alternative theories about learn- 
ers’ conceptions. These alternative theories are grounded in research in cogni- 
tive science and learners’ epistemologies. The more contemporary research (e.g.. 
Strike & Posner, 1992; Pintrich, Marx, & Boyle; diSessa, Minstrell) is showing that 
the conditions of the learning environment and the motivation for learning are 
as important as the initial conceptual schemes held by the students. Simply stat- 
ed, the conceptual ecology is much more complex than originally proposed by 
Novak, Ausubel, Champagne, Klopfer, Osborne, etc. Some of the new research 
maintains that the initial conceptual schemes are not important at all. Here I am 
referring to the idea of “knowledge-in-pieces” put forth in a theoretical framework 
by Andrea diSessa and in a practical framework by Jim Minstrell. This research 
has shown that individuals’ conceptual frameworks change with regard to the con- 
ditions of the task. This perspective challenges some of the conclusions drawn 
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in the present study. For example, why would we expect a senior citizen with no 
formal education in geology or evolutionary theory to have a knowledge base 
about an emerging scientific theory about the origins of life on Earth? This expec- 
tation on the part of the author(s) is a serious shortcoming of the present paper 
because it is a shortcoming of the theories about alternative conceptions and con- 
ceptual change teaching. 

Both reviewers also noted that there were some issues concerning the 
sample size and the claims the authors were making about the generalizability 
of their findings. So, Michael and colleagues were asked to make a major revi- 
sion to the paper. Although the review process might be seen as a gate-keep- 
ing exercise by editors and/or reviewers, editors do (or should) expect 
authors to defend their submissions if they think the reviewers have missed 
the point or are mistaken. So, Bill and I waited for Michael’s response. 

Michael: The comments by the reviewers were the most insightful and 
telling of any that 1 had received in my entire career, but they were also the 
most difficult to accept and, in the end, accommodate. To say that the 
authors went back to the “drawing board” is an understatement. 

1 read new literature, attended conference presentations by people cited 
as important to consider, and constructed concept maps to integrate the new 
ideas into a more robust conceptual framework. These efforts led to a refor- 
mulation of the basic research question, moving from the traditional outcomes- 
based approach to the more complex task of understanding the experiences 
of visitors to Midway Geyser Basin. The questions evolved in this way: 

• Old: What are park visitors’ understandings about geothermal features and 
associated microorganisms? 

• New: What are the experiences of visitors to Midway Geyser Basin? 

As my attention increasingly focused on the experience of visitors, the 
new theories and principles (both of concept and methodology) the editors 
introduced to me began to make more and more sense. Figure 2 depicts my 
evolution by showing components of the old (standard text) and the new (ital- 
icized text). For example, in the conceptual framework, I had to integrate dis- 
parate ideas from various domains into a comprehensive view of learning in 
this situation— not an easy puzzle to solve! 

From the authors’ point of view, the reconceptualization of the underlying 
philosophy, theory, principles, and concepts was the hardest part of the 
rewriting. Once this had been clarified, it was clear that the data could be rein- 
terpreted in more meaningful ways. This led to different knowledge and value 
claims, which are depicted in the boxes on the right in Figure 2. Fortunately, 
there were some aspects of the research and final paper that did not change, 
such as research events, actual data, and transformative emergent themes. 
A conservative estimate of the amount of change that took place in this paper 
would be about 75%. Thankfully, the editors allowed approximately six 
months for the authors to resubmit the paper. 
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Yellowstone Visitor Study 



Figure 2. Yellowstone visitor study (Brody et al., 2002). Plain text 
indicates original ideas and framework of study; italicized text 
indicates new understandings. 


Justin: The final paper, published in 2002, is far stronger than the version 
that was originally submitted. From an editor’s point of view, Michael and col- 
leagues were exemplary in the manner in which they responded to the 
reviewers’ comments. 

Michael: The process of producing the final Yellowstone paper will remain 
a watershed event in my career. On one hand, it was always humbling to be 
led down a new trail by expert guides— that is, a trail where a part of your brain 


The Messy Process of Research 115 

















scolds you, saying “you should have traveled this before.” On the other hand, 
this article is about quieting that scolding voice, and recognizing how empow- 
ering it is to know that you can trust your colleagues to give you insights that, 
with patience and hard work, can result in a career-defining publication. 

Welcoming Internal Influence on Research Questions [Paul] 

Good research takes time to engage both theory and methodology intensively. 
Although many university programs work against this longer-term study 
through fee structures and scholarship award limitations, thoughtful inquiry 
is not so much a matter of learning the theory as learning to conceive one- 
self in terms of theory. In this case, a doctoral student’s interest shifted over 
a period of about three years, resulting in new research questions, a signif- 
icantly different dissertation product, and deep changes to the student’s own 
personal belief system (Barrett, 2005). 

Her original interest arose directly from her experience as a secondary 
teacher who coordinated and taught alternative, semester-long outdoor/envi- 
ronmental education programs. Early in her doctoral program she expressed 
a narrative interest in understanding what sustains teachers who work 
somewhat outside the regular high school program. She was curious about 
what motivates teachers (much like her) and where the energy, persistence, 
and patience had originated. Her original research questions, as she presents 
them, are as follows: 

• What does it mean to be a teacher of an intensive interdisciplinary 
outdoor/environmental education program? 

• Why do teachers of integrated programs choose to teach the way they do? 

• How did they get there? 

By mid-course, she began to engage intensively in both poststructural the- 
ory and fieldwork. The combination of conversations with one teacher in par- 
ticular, together with deeper thoughts about how teachers’ identities and sense 
of agency are produced, how teachings are discursively framed, and how life 
histories are culturally produced, led to a significant shift in research questions. 
Her new questions were positioned as a critical examination of how envi- 
ronmental education that espouses change can be understood and enacted 
within the conservative tendencies of the educational system. Thus, stories 
of experience are not simple tellings, and her evolved questions ask: 

• How do discourses of teaching, learning, and “nature” work to constrain 
and enable environmental educators? 

• How have teachers’ identities been produced by discourse? 

• How can life history research within a feminist poststructural framework 
be useful in addressing these two questions? 
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Beyond these evolving questions, she has also constructed a critically 
reflective re-engagement with her work as questions themselves: 

• What assumptions are embedded in the research questions (e.g. , about 
self, agency, knowledge)? 

• What kind of answers will the questions produce? 

These last two questions, together with St. Pierre’s (St. Pierre, 2000; St. 
Pierre & Pillow, 2000) admonition that poststructuralism needs to turn back 
on itself to investigate its own sets of assumptions, opened the study to anoth- 
er layer of examination. The study (which is represented in hypertext at 
www.porosity.ca) has now been extended to investigate ways in which 
research itself, through its often unacknowledged anthropocentric framing, 
might be limiting (successful) enactment of outdoor/environmental education. 


Methodology or Methods Surprises 

Optimally, published works should discuss the methodology, methods, and 
instruments used, but this does not always happen; methodologies may be 
the most oft-confused and neglected component (Dillon & Wals, 2006). 
Published works whose aim is to discuss the perils and pitfalls of these 
choices are also rare, but not completely absent (Raven, 2006). This section 
provides three stories of surprises discovered in the midst of research involv- 
ing methodologies and methods. One describes the unfortunate discovery that 
a “well-tested” instrument was useless, a second tells of the researcher's real- 
ization that the chosen method did not fit the question after all, and a third 
admits to a study where the frame of the study changed so often after data 
collection had begun that every type of external validity error was violated. 

The Useless "Well-Tested” Instrument [Joe] 

We are always seeking ways to ensure that our studies are well-grounded and 
rigorous. One long-practiced approach is to find an instrument measuring a 
component of what you hope to be studying, and to use it. Of course, there 
are the obvious challenges of ensuring that what is being measured is actu- 
ally the domain of what you want to know, but often there are more hidden 
challenges as well. 

For example, a graduate student was planning to use an instrument as 
part of a larger study. This instrument was comprised of several scales, had 
been used in at least 1 2 studies, and had good reliability measures in each 
of the studies (ranging from around a 0.69 to a 0.82). However, as we 
began the necessary work to translate the instrument, we found that sever- 
al items could be considered positive or negative, other items were neutral- 
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ly stated, and some were clearly directionally framed. What a shock that this 
instrument had been replicated and used in other studies, but had con- 
struction errors! The reliability measures were strong because there was a clear 
inter-item relationship, but the validity of the questioning structure would be 
questionable, as would any analysis on the findings. 

Although this original instrument was flawed for our use, sometimes going 
back to the original instrument is far more valuable than creating a new one 
or even using a more contemporary adaptation. For example, we’ve found 
far stronger reliability and distributions in populations using some instruments 
developed by psychologists measuring a particular construct, as compared 
to environmental education instruments measuring the same construct. As 
a second example, I have found some basic state and trait measures that are 
better than those that have been adapted specifically for environmental 
education and science. 

in sum, caution is urged both when modifying existing instruments, 
where errors can be amplified through repetition without critical reflection, 
and in using existing ones without change, where application to a new field 
must be carefully considered. 

Evolving Epistemology, Methodology, and Methods [Paul] 

In this case, a sticking point in data analysis led a veteran teacher to not only 
a change in methods or even methodology, but in fact an epistemological shift 
in what counted as legitimate data (not necessarily in that order). The setting 
for this story is the secondary school physics classroom (not the environmental 
education classroom), but the story and its outcome are well-applied to the 
environmental education field. 

The teacher in this case developed his doctoral work out of a persistent 
conundrum in his work with students on vector mathematics (Wessel, 
1998). The fact that many high school students experience difficulty in the 
transition from concrete scenario (e.g., two soccer players each kick a soc- 
cer ball simultaneously from different angles) to abstract math symbols 
and graphical representations, led him to question how adolescents “learn” 
to conceptualize complex concepts. He came to investigate students’ lived 
experiences in physics problem-solving by creating classroom conditions 
where students could verbalize their learning experiences interactively with 
peers and the researcher-as-teacher in an attempt to make their reasoning 
processes more explicit. Although difficult, he was able to collect rich narrative 
text through many days of problem-solving. 

So while context question and research process were clear enough, the 
sticking point in this study concerned the method of analysis. What to do with 
mounds of narrative text is a relatively common dilemma in many emergent 
interpretive research studies. In this case, given a background in scientific 
research, the student’s first instinct was to organize the data by dwelling inside 
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the text and looking for regularities as patterns or themes. He elected to code 
the transcripts of his data using an elaborate colour-coding scheme — 
attempting to make sense of the interactive (reasoning) talk. In my discussions 
with him during this process, questions arose about what counts as legitimate 
data and data analysis in various forms of inquiry (quantitative and qualita- 
tive). Despite rhetoric to the contrary, it was difficult for this 
teacher/researcher of science to make the interpretive turn — from viewing data 
in terms of categories to viewing it differently, perhaps holistically, so that the 
actual content of the conversations could be brought into sharper relief. 

After about three months of coding, he abandoned that analytic process 
for a form of conversation analysis within a more interpretive frame. 1 
believe the shift was analogous to his physics students’ insight into the 
transition from concrete to abstract thought in solution of their vector prob- 
lems. A highly independent person with a mind of his own, who would not 
be convinced of method except through his own experience, produced a the- 
sis rich in conversation analysis. This case, for me, represents an example of 
distinctive shifts in methodological thinking brought on by a sticking point 
in analysis that had ramifications far beyond method— in fact, in order for a 
shift in method of analysis to occur, an epistemological shift in what count- 
ed as legitimate data was prerequisite. Such shifts in thinking and data 
analysis methods are important to consider for questions that are at the con- 
crete/abstract border, many of which exist in environmental education, 
such as the study of instruction on global warming. 

The Case of the Disappearing Frame [Joe] 

In a national study, several organizations provided names from which a 
random sample would be generated. Each group was part of a blocked 
sample of these organizations and was selected to represent that type of 
organization based on size of members on their lists. Initially, each group was 
asked to provide the number of names on their list. The total number was 
determined and a proportional sample number was assigned to each group. 
The result was an initial frame, or list of accessible respondents, of around 900 
names. 

Because the study was under a strict timeline and the volunteer-led groups 
did not respond in a timely manner, the study began as soon as the first frame 
was received. This first list, when received, included duplicate names and mul- 
tiple ways of addressing the same person for some businesses, organizations, 
and individuals. The total N for the study was adjusted, the proportions 
rerun, the sample drawn, and surveys sent. The second frame was larger than 
the first, but also had many frame errors. And so it went through all 10 frames. 
Each time a list arrived it was edited, corrected, and the numbers recalculated. 
One list of 600 names was actually under 300 discrete names, and another 
list had over 60 names that were corporate sponsors rather than members. 
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Clearly, we had a problem. We (the researchers) were in a perpetual state 
of panic each time a new list of names arrived and differed from the earlier 
promised list. This created a multitude of emergent threats to validity, 
among them: 

• differential selection-experimental variable interaction, 

• selection-maturation interaction effects, 

• maturation, 

• differential selection of subjects, and 

• instability (see, for example, Smith, 1980). 

The study could have been discarded, but as most of the threats related 
to interaction effects and differentiation among the populations, all respon- 
dents were considered as one pool. The sample was changed from a blocked 
design to a single population with no blocks. In doing this, we were able to 
eliminate two of the major threats. To address the instability threat, signifi- 
cance measures were done using ANOVAs in order to look within and 
between groups, with the groups being defined by demographics rather 
than the block design. Finally, the threat of maturation was addressed by look- 
ing at differences between late respondents from one frame and comparing 
them to early respondents of the next frame. 

The final result: a national study of a random selection of individuals from 
10 organizations that could be considered representative of the larger pop- 
ulation. Statistics were nonparametric or, in the case of the ANOVAs, para- 
metric but for the purpose of more clearly describing the population, rather 
than generalizing. Although the study was published in peer-reviewed jour- 
nals without a hint of the frame error, the real learning for me was the chal- 
lenge of addressing the compounding problems. 


Problematic Answers 

In theory, studies ask questions or state hypotheses without knowing the 
answer(s) that will be generated. In reality, most students, researchers, 
supervisors, funders, and sponsors have an opinion or wish as to what an 
answer might be. Quality study design aims to enable that wish to either be 
subsumed (as much as possible) under statistical methods, or to be acknowl- 
edged and articulated to the extent that it could impact the study findings. 
Nonetheless, stories of some problematic answers may be useful and inter- 
esting to others in the field. These stories illustrate the jeopardy of discovering 
things a research sponsor doesn’t want to know, of learning that the funder 
(or implementer) is not interested in what you have found, of losing access 
to some answers mid-stream, and of being caught in political mire. 
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Finding Answers the Funder Doesn’t Want to Flear [Joe] 

There is an ethical dilemma when doing evaluation work because the work 
belongs to the organization paying for the evaluation, rather than the evaluator. 
This can lead to real problems when someone agrees they want an evalua- 
tion to “find out what’s working and what can be changed to improve the pro- 
gram” upfront, but when the report is completed decides that they want every- 
thing to look good for the funder. On numerous occasions, this shift has led 
to the organization asking to remove any of the findings that are even 
slightly critical, and all recommendations for improvement. Two alternate 
approaches could be tried: 

• provide the report and allow the organization to pull from the report those 
things they choose to share, but do not allow alteration of the report itself; or 

• offer to present the findings verbally to the funder. 

The former creates additional tension around the findings and can lead to 
resistance to good evaluation. The latter takes additional time, but demon- 
strates to the program people how to manage findings with funders. It also 
helps the funders see why a program is truly worthy of additional or continued 
support, as it is continually improving. 

In one study, a board member did not want membership of the organi- 
zation studied as part of the larger organizational review, and blocked progress 
on the study. Repeated discussions and messages to stop gathering data led 
the study team to realize that this individual feared members would not be in 
favour of some pet projects (changes) of this board member. To satisfy the 
member and remain ethically true to the research (which needed the input of 
members), members were not surveyed, but a random selection from the 
membership roster was interviewed by telephone. This kept the member com- 
ponent of the study separate from the study of staff, board, administration, 
and the community at large. Not surprisingly, some of the members inter- 
viewed liked the changes, and some didn’t. However, major discontent was 
expressed by members over the way the board and administrators were 
handling the changes. Members felt that too many important things happening 
were being kept from them, as exemplified by the reluctance to include 
members in the study itself! Although this observation was not included in the 
final report, the findings were presented and one bullet point noted the dis- 
satisfaction of the membership. In this way, we feel we were able to include 
the concept without being confrontational and/or risking that key stakehold- 
ers would reject the entire document because they didn’t like one piece. 

Murky Participant Permission Process [Charlotte] 

In a study of a group of people planning to build (and live in) a neighbourhood, 
I was granted initial permission for the research by the community through 
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a consensus decision process (and a permission form signed subsequently 
by community leaders). This permission permitted me to attend communi- 
ty meetings, have access to neighbourhood documents, and participate in the 
neighbourhood Internet list serve. It also allowed me to invite individuals to 
participate in interviews (1 obtained separate written permission on an indi- 
vidual basis). About half the households in the community agreed to (and par- 
ticipated in) an individual interview. Notwithstanding the fact that this study 
was exempted from the review of the Institutional Review Board at my 
institution, 1 have made regular and thorough attempts throughout the years 
to re-ascertain permission from the community as a whole, and with indi- 
viduals (especially newcomers) in the community. 

After many years of data collection and analysis of these data, I found that 
some of the most interesting stories and findings feature members of the 
neighbourhood who did not elect to participate in an individual interview. This 
produced a dilemma for the research— some findings of high potential 
interest to the study existed in a murky access situation. These individuals 
had agreed, through a group consensus decision, to being observed at meet- 
ings and to having their emails read that they posted to the list serve. 
However, 1 believe that their lack of permission for an individual interview 
might indicate a discomfort with a more public illumination of their partic- 
ular participation. Therefore, if I want to highlight those stories and to 
remain comfortable that permission has been granted, 1 must decide how, 
when, and whether to approach these individuals to obtain a second level of 
permission. A risk exists that a participant may decline permission, which 
could have implications for many levels of data. Therefore, the decision is actu- 
ally whether to put some of the data at risk of becoming unusable (with 
unknown “domino-style” implications), or to use other data (featuring those 
participants who have given individual interviews) to reveal similar or different 
findings. In the one case that inspired this anecdote, 1 asked for and was grant- 
ed permission to tell the story (“I didn’t agree to an interview because I was 
just busy,” she said.) Nonetheless, I wonder whether in the end, I will believe 
that my initial and ongoing permission process was not thorough enough, or 
that my data and findings were better and richer as a result of the less inva- 
sive approach. 


Publication Dilemmas 

Many experienced and aspiring academics can tell stories of trying to guess 
unsuccessfully what is in the editor’s head or, when a piece is rejected from 
a journal, knowing when to work to change the article to fit, and when (and 
how) to submit the piece to a different journal where it might fit better. 
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Resubmitting to the Same Journal [Justin] 

Different journals operate different review procedures. The International 
Journal of Science Education, for example, has seen the number of submissions 
rise, particularly with the introduction of an electronic system (Manuscript 
Central). The electronic system obviates the need for posting printed papers 
across the globe which, as well as having environmental benefits, also has con- 
comitant time savings. The journal now publishes 15 editions each year and 
editors can afford to be robust in attempting to raise the quality of the jour- 
nal by sending papers to two or more experienced researchers. It is extreme- 
ly unusual for a paper to be accepted without any revisions. In my experience, 
almost all papers are either rejected (with an encouragement to resubmit) or 
they are sent back for major revisions. The message is: persist, listen to the 
reviewers’ comments, and be prepared for a lot of extra work after you think 
you’ve finished a paper! 

Resubmitting to a Different Journal [Michael] 

Sometimes, a paper may not need rewriting as much as it may need resub- 
mitting to a different journal. The reason can be a lack of fit with the type of 
work the journal publishes, or it may be a more amorphous or invisible rea- 
son. In the case of the former, an author should try to identify a journal’s 
theme before submission and, where a misfit occurs, hope that comments 
from the journal will steer the article in a better direction. Two stories of the 
latter (amorphous or invisible reasons for rejection) follow. 

In one case, an editor rejected an article of mine out-of-hand, and with- 
out review, because it was based on a well-known author’s perspective. 
Specifically, I had taken the author’s perspective on local, social, cultural, and 
political aspects of a watershed and its potential role in guiding ecological gov- 
ernance and sustainability, and had related it to place-based curriculum 
development in environmental education. Apparently, the journal editors 
believed the application of the perspective to a different curricular situation 
was not unique enough to warrant publication. I chose not to revise but to 
resubmit to a different publication, where the article was published without 
any editorial changes. 

In a second case, 1 realized that several papers from early in my career 
could be combined into a synthesis paper focusing on student understand- 
ing of ecological crises. The new paper addressed science and natural 
resource management concepts across several ecological phenomena and pro- 
vided insight across disciplines. As with the example above, the first sub- 
mission was again rejected out-of-hand, and was not sent out for review by 
contributing editors. The only reason given was that the research involved 
work over five years with the assistance of approximately 30 graduate stu- 
dents. Again, a subsequent submission of the same paper to an international 
journal resulted in publication with little revision. 
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These two experiences have helped me develop a “thick skin” when it 
comes to reviews that seem to simply dismiss my work. My current per- 
spective is that editors have power that they may choose to wield in a vari- 
ety of ways, some not as fair or just as others. From my experience, it 
appears second opinions are just as good in publishing as they are in med- 
ical diagnoses. 


Conclusion 

This article is about encouragement. We identify with the challenges researchers 
face in exploring their questions and interests in the face of new theoretical 
perspectives— interpretive, critical, and postmodern. More traditional preor- 
dinate frames have been supplanted. In their place, new kinds of questions 
require diverse theoretical perspectives that often cross disciplinary boundaries. 
Indeed, we have gone beyond the days when a cookbook approach to meth- 
ods was sufficient. Researchers now employ a range of methods across a wide 
spectrum of methodologies that advocate emergent methods more con- 
sciously responsive to changing conditions in the field, as well as sensitive to 
theoretical issues of ethics, power, and authority. The issues, methodology, and 
methods needs cannot always be anticipated in design. 

The authors have attempted to provide insights from work amongst 
diverse categories and frames of inquiry that are meant to encourage fellow 
students to persist and problem-solve mid-process. We encourage an emer- 
gent trend for authors to write about, and publish, their own struggles in a way 
that anticipates critique. We hope that increasing instances of researchers 
telling stories of mucking around the methodologies and ethics of inquiry can 
be the beginning of a more process-oriented, and perhaps more nuanced con- 
versation that opens active discussion and scrutiny amongst critical friends. 
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